Advanced lane lines finding

In this project, your goal is to write a software pipeline to identify the lane boundaries in a video from a front-facing camera on a car.

Step1: Camera Calibration with OpenCV

Run the code in the cell below to extract object points and image points for camera calibration.

Geometric camera calibration, also referred to as camera resectioning, estimates the parameters of a lens and image sensor of an image or video camera. You can use these parameters to correct for lens distortion, measure the size of an object in world units, or determine the location of the camera in the scene.

In [27]:
In [2]:
Calibration image shape is:  (720, 1280, 3)
In [4]:

Here, I will do camera calibration (Note: this step should be done only once)

In [5]:
(720, 1280, 3)
 Camera Calibration done
In [6]:
Out[6]:
Text(0.5,1,'Undistorted Image')

In the next steps, up to step 8, I will apply every step and see the output on a test image, then after i will build my pipeline function and apply on all supported test images and then on the video.

Step 2: Undistort and warp the image (Perspective transform)

Here I will apply distortion correction and warp the image to get bird's view of the image

In [2]:
end
In [8]:
(720, 1280, 3)

Step2: Apply combind thresholds to our un-distorted and warped image, and get a combined binary image as shown below.

Gradient threshold + Gradient magnitude threshold + Gradient direction threshold

In [3]:
In [10]:
In [11]:
In [12]:

Step 3: Color and Gradient threshold

In this step, I will apply combined Gradient threshold on a filtered image version of the original image, here as suggested, I will use and s-channel filtered image and then will test on both normal image and undist+warped image as in Step2, testing part.

In [4]:
In [14]:
In [15]:

Step 4: In this step I will build my pipeline

Here I will merge all previous steps in one pipeling fuction and apply on all test images

In [11]:
end
In [40]:
Out[40]:
<matplotlib.image.AxesImage at 0x9e93198>
In [18]:
Out[18]:
['straight_lines1.jpg',
 'straight_lines2.jpg',
 'test1.jpg',
 'test2.jpg',
 'test3.jpg',
 'test4.jpg',
 'test5.jpg',
 'test6.jpg']
In [19]:
lenght of outputs :  8
In [20]:
length of images to display =  16

As shown above, images with lots of shadows effect, hold alot of noise in the binary version, as in images 6 & 7.

So, I will add to the pipeline, another color threshold. Since, R-channel still does good job on white lines, perhaps better than S-channel. So will combine it with the Binary image from the pipeline and test.

Adding R-Channel threshold.

In [5]:
In [32]:
Out[32]:
Text(0.5,1,'Pipeline Image with R_channel')

As shown above, the image doesn't show big difference even with changing in the threshold values!, so I will now try the L-Channel threshold and see the effect.

In [6]:
In [34]:
Out[34]:
Text(0.5,1,'Pipeline Image with L_channel')

Now I will test the pipeline on all images (new pipeline only uses combined R and L trhesholds )

In [7]:
In [49]:

As shown above, the shadows and lightness effect is getting better after combining both R-channel and L-channel threshold more than it was when using S-channel. No as shown in the last 2-images (image 6,7), we still have problem with yellow lines. as suggested here, to best grab the yellow color is shifting to LAB color space and filt B-channel, so I will add the b-channel() function to the pipeline and test. since B-channel shows how yellow or blue the colour is.

In [13]:
In [54]:

As shown above, I can't see a big enhancement in combining R-L-B channels more than just using R-L channels,so now I will do some expirement to get the best combination of channels

In [72]:

Step 5: Finding the lane lines and fit polynomial

Line Finding Method: Peaks in a Histogram

In [160]:
Out[160]:
[<matplotlib.lines.Line2D at 0x10526a58>]
In [86]:
Out[86]:
<matplotlib.image.AxesImage at 0x9e91588>

Implement Sliding Windows and Fit a Polynomial

Since I found that ther are some misleading information on the right of the image, I decided to crop a small part of the right of the image, as shown in the following funciton

In [14]:
(720, 1100)
Out[14]:
<matplotlib.image.AxesImage at 0x9f66358>
In [15]:
end
In [16]:
In [93]:
Out[93]:
[<matplotlib.lines.Line2D at 0x145361d0>]

As shown above, for the image I am testing here, which is (test2.jpg), as the histogram peaks correctly reflects the lane lines, so the function lane_lines(), correctly draws them, however in (test1.jpg), since historgram shows peaks on the right side of the road not the lane lines, the functiion lane_lines() didn't work correctly as shown bellow.

In [114]:
In [116]:
Out[116]:
[<matplotlib.lines.Line2D at 0x105e20b8>]
In [ ]:

Skip the sliding windows step once you know where the lines are

In [17]:
end
In [18]:
end
In [151]:
Out[151]:
(720, 0)
In [143]:
In [19]:
In [155]:

As shown above, all lane lines are correctly detected and drawn excetp for the 1st and 3rd images, as ther histogram shows peaks on the right side of the road instead of the lane line!!! so, I tried in the begining of my algorithm use the histogram of the bottom 1/3 part of the image instead of the bottom 1/2, and it helped in detecting the lane-lines correctly for all images. As shown below:

In [162]:

Step 6: Measuring lane curvature and Vehicle position w.r.t center

In [20]:
end
In [228]:
left curvature radius=  1576.19975871 m 
right curvature radius=  4265.89670869 m 
car position w.r.t.center=  0.798783884219 m

For the position of the vehicle w.r.t center point, I assumed that the camera is perfectly located at the middle of the car, and the deviation of the mid-point of the lane from the center of the image is considered the position of the vehicle w.r.t. center point.

Step 7: Warp detected lanes back to original image.

In [21]:
In [199]:
Out[199]:
Text(0.5,1,'un-warped image')
In [22]:
end
In [233]:
left curvature radius=  1576.19975871 m 
right curvature radius=  4265.89670869 m 
car position w.r.t.center=  0.798783884219 m

Step 8: Output visual display of the lane boundaries and numerical estimation of lane curvature and vehicle position.

This step is done in the above piece of code.

Step 9: Writing a complete pipeline

In [23]:
end

Step 10: Testing on Videos

In [25]:
In [235]:
[MoviePy] >>>> Building video output_videos/project_video.mp4
[MoviePy] Writing video output_videos/project_video.mp4
100%|█████████████████████████████████████▉| 1260/1261 [05:50<00:00,  3.55it/s]
[MoviePy] Done.
[MoviePy] >>>> Video ready: output_videos/project_video.mp4 

Wall time: 6min 2s
In [ ]:
Save